Fragmentation in trader preferences among multiple markets: market coexistence versus single market dominance.

Technological advancement has led to an increase in the number and type of trading venues and a diversification of goods traded. These changes have re-emphasized the importance of understanding the effects of market competition: does proliferation of trading venues and increased competition lead to dominance of a single market or coexistence of multiple markets? In this paper, we address these questions in a stylized model of zero-intelligence traders who make repeated decisions at which of three available markets to trade. We analyse the model numerically and analytically and find that the traders' decision parameters-memory length and how strongly decisions are based on past success-make the key difference between consolidated and fragmented steady states of the population of traders. All three markets coexist with equal shares of traders only when either learning is too weak and traders choose randomly, or when markets are identical. In the latter case, the population of traders fragments across the markets. With different markets, we note that market dominance is the more typical scenario. Overall we show that, contrary to previous research emphasizing the role of traders' heterogeneity, market coexistence can emerge simply as a consequence of co-adaptation of an initially homogeneous population of traders.

Dynamical selection of Nash equilibria using reinforcement learning: Emergence of heterogeneous mixed equilibria.

We study the distribution of strategies in a large game that models how agents choose among different double auction markets. We classify the possible mean field Nash equilibria, which include potentially segregated states where an agent population can split into subpopulations adopting different strategies. As the game is aggregative, the actual equilibrium strategy distributions remain undetermined, however. We therefore compare with the results of a reinforcement learning dynamics inspired by Experience-Weighted Attraction (EWA) learning, which at long times leads to Nash equilibria in the appropriate limits of large intensity of choice, low noise (long agent memory) and perfect imputation of missing scores (fictitious play). The learning dynamics breaks the indeterminacy of the Nash equilibria. Non-trivially, depending on how the relevant limits are taken, more than one type of equilibrium can be selected. These include the standard homogeneous mixed and heterogeneous pure states, but also heterogeneous mixed states where different agents play different strategies that are not all pure. The analysis of the reinforcement learning involves Fokker-Planck modeling combined with large deviation methods. The theoretical results are confirmed by multi-agent simulations.

Stochastic evolution in populations of ideas.

It is known that learning of players who interact in a repeated game can be interpreted as an evolutionary process in a population of ideas. These analogies have so far mostly been established in deterministic models, and memory loss in learning has been seen to act similarly to mutation in evolution. We here propose a representation of reinforcement learning as a stochastic process in finite 'populations of ideas'. The resulting birth-death dynamics has absorbing states and allows for the extinction or fixation of ideas, marking a key difference to mutation-selection processes in finite populations. We characterize the outcome of evolution in populations of ideas for several classes of symmetric and asymmetric games.